Robots operating in real-world environments frequently encounter unknown objects with complex structures and articulated components, such as doors, drawers, cabinets, and tools. The ability to perceive, track, and manipulate these objects without prior knowledge of their geometry or kinematic properties remains a fundamental challenge in robotics. In this work, we present a novel method for visuo-tactile-based tracking of unseen objects (single, multiple, or articulated) during robotic interaction without assuming any prior knowledge regarding object shape or dynamics. Our novel pose tracking approach termed ArtReg (stands for Articulated Registration) integrates visuo-tactile point clouds in an unscented Kalman Filter formulation in the SE(3) Lie Group for point cloud registration. ArtReg is used to detect possible articulated joints in objects using purposeful manipulation maneuvers such as pushing or hold-pulling with a two-robot team. Furthermore, we leverage ArtReg to develop a closed-loop controller for goal-driven manipulation of articulated objects to move the object into the desired pose configuration. We have extensively evaluated our approach on various types of unknown objects through real robot experiments. We also demonstrate the robustness of our method by evaluating objects with varying center of mass, low-light conditions, and with challenging visual backgrounds. Furthermore, we benchmarked our approach on a standard dataset of articulated objects and demonstrated improved performance in terms of pose accuracy compared to state-of-the-art methods. Our experiments indicate that robust and accurate pose tracking leveraging visuo-tactile information enables robots to perceive and interact with unseen complex articulated objects (with revolute or prismatic joints).

Learning to manipulate objects efficiently, particularly those involving sustained contact (e.g., pushing, sliding) and articulated parts (e.g., drawers, doors), presents significant challenges. Traditional methods, such as robot-centric reinforcement learning (RL), imitation learning, and hybrid techniques, require massive training and often struggle to generalize across different objects and robot platforms. We propose a novel framework for learning object-centric manipulation policies in force space, decoupling the robot from the object. By directly applying forces to selected regions of the object, our method simplifies the action space, reduces unnecessary exploration, and decreases simulation overhead. This approach, trained in simulation on a small set of representative objects, captures object dynamics -- such as joint configurations -- allowing policies to generalize effectively to new, unseen objects. Decoupling these policies from robot-specific dynamics enables direct transfer to different robotic platforms (e.g., Kinova, Panda, UR5) without retraining. Our evaluations demonstrate that the method significantly outperforms baselines, achieving over an order of magnitude improvement in training efficiency compared to other state-of-the-art methods. Additionally, operating in force space enhances policy transferability across diverse robot platforms and object types. We further showcase the applicability of our method in a real-world robotic setting. For supplementary materials and videos, please visit: https://tufts-ai-robotics-group.github.io/FLEX/

Robots generally have a structure that combines rotational joints and links in a serial fashion. On the other hand, various joint mechanisms are being utilized in practice, such as prismatic joints, closed links, and wire-driven systems. Previous research have focused on individual mechanisms, proposing methods to design robots capable of achieving given tasks by optimizing the length of links and the arrangement of the joints. In this study, we propose a method for the design optimization of robots that combine different types of joints, specifically rotational and prismatic joints. The objective is to automatically generate a robot that minimizes the number of joints and link lengths while accomplishing a desired task, by utilizing a black-box multi-objective optimization approach. This enables the simultaneous observation of a diverse range of body designs through the obtained Pareto solutions. Our findings confirm the emergence of practical and known combinations of rotational and prismatic joints, as well as the discovery of novel joint combinations.

This work stages Foosball as a versatile platform for advancing scientific research, particularly in the realm of robot learning. We present an automated Foosball table along with its corresponding simulated counterpart, showcasing a diverse range of challenges through example tasks within the Foosball environment. Initial findings are shared using a simple baseline approach. Foosball constitutes a versatile learning environment with the potential to yield cutting-edge research in various fields of artificial intelligence and machine learning, notably robust learning, while also extending its applicability to industrial robotics and automation setups. To transform our physical Foosball table into a research-friendly system, we augmented it with a 2 degrees of freedom kinematic chain to control the goalkeeper rod as an initial setup with the intention to be extended to the full game as soon as possible. Our experiments reveal that a realistic simulation is essential for mastering complex robotic tasks, yet translating these accomplishments to the real system remains challenging, often accompanied by a performance decline. This emphasizes the critical importance of research in this direction. In this concern, we spotlight the automated Foosball table as an invaluable tool, possessing numerous desirable attributes, to serve as a demanding learning environment for advancing robotics and automation research.

This paper introduces the Visual Inverse Kinematics problem (VIK) to fill the gap between robot Inverse Kinematics (IK) and visual servo control. Different from the IK problem, the VIK problem seeks to find robot configurations subject to vision-based constraints, in addition to kinematic constraints. In this work, we develop a formulation of the VIK problem with a Field of View (FoV) constraint, enforcing the visibility of an object from a camera on the robot. Our proposed solution is based on the idea of adding a virtual kinematic chain connecting the physical robot and the object; the FoV constraint is then equivalent to a joint angle kinematic constraint. Along the way, we introduce multiple vision-based cost functions to fulfill different objectives. We solve this formulation of the VIK problem using a method that involves a semidefinite program (SDP) constraint followed by a rank minimization algorithm. The performance of this method for solving the VIK problem is validated through simulations.

Inverse kinematics (IK) is a fundamental problem frequently occurred in robot control and motion planning. However, the problem is nonconvex because the kinematic map between the configuration and task spaces is generally nonlinear, which makes it challenging for fast and accurate solutions. The problem can be more complicated with the existence of different physical constraints imposed by the robot structure. In this paper, we develop an inverse kinematics solver named IKSPARK (Inverse Kinematics using Semidefinite Programming And RanK minimization) that can find solutions for robots with various structures, including open/closed kinematic chains, spherical, revolute, and/or prismatic joints. The solver works in the space of rotation matrices of the link reference frames and involves solving only convex semidefinite problems (SDPs). Specifically, the IK problem is formulated as an SDP with an additional rank-1 constraint on symmetric matrices with constant traces. The solver first solves this SDP disregarding the rank constraint to get a start point and then finds the rank-1 solution iteratively via a rank minimization algorithm with proven local convergence. Compared to other work that performs SDP relaxation for IK problems, our formulation is simpler, and uses variables with smaller sizes. We validate our approach via simulations on different robots, comparing against a standard IK method.

In this paper, we review and compare several velocity-level and acceleration-level Pseudo-Inverse-based Path Planning (PPP) and Pseudo-Inverse-based Repetitive Motion Planning (PRMP) schemes based on the kinematic model of robotic manipulators. We show that without unit consistency in the pseudo-inverse computation, path planning of incommensurate robotic manipulators will fail. Also, we investigated the robustness and noise tolerance of six PPP and PRMP schemes in the literature against various noise types (i.e. zero, constant, time-varying and random noises). We compared the simulated results using two redundant robotic manipulators: a 3DoF (2RP), and a 7DoF (2RP4R). These experimental results demonstrate that the improper Generalized Inverse (GI) with arbitrary selection of unit and/or in the presence of noise can lead to unexpected behavior of the robot, while producing wrong instantaneous outputs in the task space, which results in distortions and/or failures in the execution of the planned path. Finally, we propose and demonstrate the efficacy of the Mixed Inverse (MX) as the proper GI to achieve unit-consistency in path planning.

Recent advancements in legged locomotion research have made legged robots a preferred choice for navigating challenging terrains when compared to their wheeled counterparts. This paper presents a novel locomotion policy, trained using Deep Reinforcement Learning, for a quadrupedal robot equipped with an additional prismatic joint between the knee and foot of each leg. The training is performed in NVIDIA Isaac Gym simulation environment. Our study investigates the impact of these joints on maintaining the quadruped's desired height and following commanded velocities while traversing challenging terrains. We provide comparison results, based on a Cost of Transport (CoT) metric, between quadrupeds with and without prismatic joints. The learned policy is evaluated on a set of challenging terrains using the CoT metric in simulation. Our results demonstrate that the added degrees of actuation offer the locomotion policy more flexibility to use the extra joints to traverse terrains that would be deemed infeasible or prohibitively expensive for the conventional quadrupedal design, resulting in significantly improved efficiency.

Soft robots are promising for manipulation tasks thanks to their compliance, safety, and high degree of freedom. However, the commonly used bidirectional continuum segment design means soft robotic manipulators only function in a limited hemispherical workspace. This work increases a soft robotic arm's workspace by designing, fabricating, and controlling an additional soft prismatic actuator at the base of the soft arm. This actuator consists of pneumatic artificial muscles and a piston, making the actuator back-driveable. We increase the task space volume by 116\%, and we are now able to perform manipulation tasks that were previously impossible for soft robots, such as picking and placing objects at different positions on a surface and grabbing an object out of a container. By combining a soft robotic arm with a prismatic joint, we greatly increase the usability of soft robots for object manipulation. This work promotes the use of integrated and modular soft robotic systems for practical manipulation applications in human-centered environments.

This paper addresses the task of category-level pose estimation for articulated objects from a single depth image. W e present a novel category-level approach that correctly accommodates object instances not previously seen during training. A key aspect of the work is the new Articulation-Aware Normalized Coordinate Space Hierarchy (A-NCSH), which represents the different articulated objects for a given object category. This approach not only provides the canonical representation of each rigid part, but also normalizes the joint parameters and joint states. W e developed a deep network based on PointNet that is capable of predicting an A-NCSH representation for unseen object instances from single depth input. The predicted A-NCSH representation is then used for global pose optimization using kinematic constraints. W e demonstrate that constraints associated with joints in the kinematic chain lead to improved performance in estimating pose and relative scale for each part of the object. W e also demonstrate that the approach can tolerate cases of severe occlusion in the observed data.